Statistics for HCI: Making Sense of Quantitative Data by Alan Dix

Author:Alan Dix
Language: eng
Format: epub
Publisher: Morgan & Claypool Publishers
Published: 2020-07-14T16:00:00+00:00

7.4 HOW DO YOU GET THE PRIOR?

Sometimes you have strong knowledge of the prior probability, perhaps based on previous similar experiments. While this is commonly the case when Bayesian inference is used in internal algorithms, it is unlikely to be the case in more typical usability settings such as the comparison between two systems. In the latter you are usually attempting to quantify your expert judgement.

Sometimes the evidence from the experiment or study is so overwhelming that it doesn’t make much difference what prior you choose … but in such cases hypothesis testing would give very high significance levels (small p-values), and confidence intervals very narrow ranges. It is nice when this happens, but if this were always the case we would not need the statistics!

Another option is to be conservative in your prior. The first example we gave was very conservative, giving the new system a low probability of success. More commonly, a uniform prior is used, giving everything the same prior probability. This is easy when there are a small number of distinct possibilities, you just make them equal, but it’s a little more complex for unbounded value ranges, where often a Cauchy distribution is used … this is bell shaped, a bit like the Normal distribution, but has fatter edges, like a fried egg with more white.

In fact, if you use a uniform prior then the results of Bayesian statistics are pretty much identical to traditional statistics, the posterior is effectively the likelihood function, and the odds ratio is closely related to the significance level. Indeed in a re-analysis of the data from 855 psychology articles, Wetzels et al. found very little difference in using traditional or Bayesian methods [75].2

As we saw, if you do not use a uniform prior, or a prior based on well-founded previous research, you have to be very careful to avoid confirmation bias.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.